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Description 

Background of the Invention 

[0001] This invention relates to multimedia conferenc- 5 
ing via the public switched telephone network (hereafter, 
the PSTN) and data networks external to the PSTN. As 
used herein, "multimedia conferencing" means voice 
telephone conferences accompanied by presentations 
of displayable images, and the PSTN refers to equip- w 
ment throughout the world enabling users of voice tele- 
phones (private and public ones) to connect to each oth- 
er, including mobile radio systems but excluding equip- 
ment maintained or leased for exclusive use of private 
enterprises (PBX's, leased lines, etc.) as well as public 15 
data transmission systems capable of handling voice 
transmissions (e.g. the Internet and World Wide Web). 
More specifically, this invention concerns a system for 
multimedia conferencing wherein conference partici- 
pants can coordinate display of images (e.g . charts) with 20 
voice presentations, and also directly control or admin- 
ister functions or services of the PSTN that normally are 
inaccessible while a conference session is active. 
[0002] PCT patent application W095 10917 A dis- 
closes a cable network for carrying audio and video data 25 
which is overlayed with a Local Area Network (LAN). 
Switching of the audio and video signals on the cable 
network is provided by. a switchbox, controllable from 
the LAN by a server. 

[0003] European Patent Application 0 664 636 dis- 30 
closes an audio conferencing system in which speech 
from a user at a workstation is transmitted to another 
workstation and also converted into text by speech rec- 
ognition software for transmission to the other worksta- 
tion. Both the speech and the recognised text is trans- 35 
mitted over the same network. 

[0004] European Patent Application 0 659 006 dis- 
closes an audio conferencing system in which multiple 
audio input streams are received. Digital samples from 
each of those streams are multiplied by individual *o 
weighting factors before the streams are summed for 
output. A user can adjust the weighting factors for each 
of the streams, thus allowing the user to alter the relative 
volumes of each of the streams. 

[0005] Present conference management resources of *s 
the PSTN support conferencing services between voice 
telephone users wherein two or more users remote from 
one another (hereinafter, participants, conferees or at- 
tendees) can converse as if connected to a party line. 
In addition to overseeing connections between partici- 50 
pants, these management resources control other con- 
ference parameters, such as the maximum number of 
participants to be accommodated, the maximum dura- 
tion of the session, connection of new participants, no- 
tification of participants when a new participant is added, 55 
etc. 

[0006] The present infrastructure of the PSTN also al- 
lows for direct transmission of data between PSTN sub- 



scribers (e.g. facsimile data, computer data, etc.), and 
may be used to support transmission of data between 
parts of global data networks like the Internet and World 
Wide Web (hereafter, the Web or WWW). Parts of such 
data networks are physically and logically external to the 
PSTN in that they do not form or take part in forming 
end-to-end connections between ordinary voice tele- 
phones in response to direct dialling (or tone keying). 
Parts external to the PSTN include for instance equip- 
ment maintained by Internet access providers, which is 
not subject to control by the PSTN, even when conduct- 
ing transmissions over the data network and even if con- 
nected to the PSTN while doing so. Notably, in addition 
to lacking control of such external parts, the PSTN has 
no control over the routing of transmissions between 
them via transmission lines in the data network. 
[0007] US patent application serial number US-A- 
8/761,257 entitled USER INVOCATION OF SERVICES 
IN PUBLIC SWITCHED TELEPHONE NETWORK VIA 
PARALLEL DATA NETWORKS discloses a telephone 
call management system for a PSTN (public switched 
telephone network) enabling users of that network to in- 
voke or disable services (e.g. call waiting, call blocking, 
call forwarding, etc.) via public data networks like the 
Internet, and via user actions independent of telephone 
call activity; e.g. while a telephone connection is active 
or while the user's telephone is idle (on-hook). 
[0008] Furthermore, even if voice signals are trans- 
mitted through the data network, the PSTN has no con- 
trol over their routing within the data network. Hence, 
even parts of the Internet/Web that may carry voice sig- 
nals are external to the PSTN, even if the signals are 
generated and/or reproduced in telephones or tele- 
phone-like apparatus (e.g. microphones and speakers 
connected to computers and conveyed through mo- 
dems), if the PSTN can not control those parts. 
[0009] Similarly, private networks containing trans- 
mission lines leased from the PSTN (T1 lines, T2 lines, 
etc.), may include parts that are not subject to PSTN 
control, which therefore are external to the PSTN re- 
gardless of what signals they carry (data, voice, or oth- 
er). 

[0010] Consequently, users of PSTN conferencing 
services can not presently operate through either exter- 
nal networks or their telephones to administer complex 
services within the PSTN; particularly, while a confer- 
ence is in session. 

[001 1] The present invention seeks to overcome con- 
ferencing limitations of both the PSTN and networks ex- 
ternal thereto (particularly, public data networks like the 
Internet and Web), to enable participants in a multime- 
dia conference to use external networks to vary services 
received in the PSTN during the conference; without re- 
quired intervention of PSTN operators or other PSTN 
representatives. This is accomplished by outfitting the 
PSTN with conference servers that are administered 
and controlled by the PSTN but have links to external 
networks. The conference servers are so constructed 



20 



25 



30 



35 



40 



45 



50 



2 



3 



EP 0 847 178 B1 



4 



and located as to enable participants in a multimedia 
conference to control distribution of display images to 
other participants through public data networks (e.g. the 
Internet) in coordination with their voice presentations, 
and to permit such control at a cost that can be attractive 
to both the PSTN and its customers. The conference 
servers are also so constructed and configured within 
the PSTN as to allow participants in a multimedia con- 
ference to control services internal to the PSTN (both 
existing services such as conference outdialling and 
new ones) by means of signal requests communicated 
through external networks. 

Summary of The Invention 

[0012] It is an object of the present invention to pro- 
vide a technique which alleviates the above drawbacks. 
[001 3] According to the present invention we provide 
a voice and data conferencing system as specified in 
claim 1. 

[0014] During a multimedia conference, voice (or oth- 
er audio) signals pass between conferees through con- 
ventional switch circuits and conference bridge circuits 
that are internal to the PSTN and that are administered 
by conventional conference management resources of 
the PSTN. However, data signals in the same confer- 
ence (image data and other data) may be routed via a 
conference server over a public data network external 
to the PSTN, like the Internet or Web. Each such con- 
ference server is administered and controlled by the 
PSTN and has links to conference connection and con- 
ference management resources of the PSTN, such that 
a conference participant making a voice presentation 
can furnish image data to the conference server during 
or prior to the presentation, have the data stored in the 
conference server, and then have the data uploaded 
from the conference server to the other participants in 
precise coordination with parts of trie voice presentation 
referring specifically to respective display images. Con- 
nections between conference participants and public 
data networks linked to a conference server may be di- 
alled* (tone keyed) telephone system connections or di- 
rect connections through e.g. privately maintained lines 
and/or networks. 

[0015] Each conference server and its aforesaid links 
within the PSTN is also useful to allow conferees to con- 
trol operational services within both the PSTN and the 
external data network, both before and during a confer- 
ence. 

[001 6] In respect to coordinated transfer of image da- 
ta, a conference server storing data furnished by a con- 
ference participant can be requested by that participant 
to distribute the data to other conference participants. 
The request can be sent through either the external data 
network (e.g. as a data signal representing the request) 
or through the PSTN conference connection as spoken 
commands if the PSTN is suitably equipped for detect- 
ing and reacting to such commands. 



[001 7] In respect to user control of conditions or serv- 
ices internal to the PSTN, such control can be used to 
interrupt a speaker in the conference; e.g. by a request 
over the data network causing the conference server to 

5 have images displayed to conferees (including the 
speaker) modified to show an indication requesting in- 
terruption of the speaker (e.g. a displayed indication 
connoting QUESTION TO ASK or NEED TO TALK); 
thereby allowing for orderly interruption of speakers 

10 without having plural participants trying to speak at the 
same time. 

[0018] Another use of this facility would be to allow a 
conferee to request the conference server to adjust am- 
plitude or other characteristics of audio signals currently 
15 being received by that conferee; whereby, for instance, 
a hearing impaired conferee could raise the volume of 
their received audio without affecting audio volume re- 
ceived by other participants. 

[0019] Another use would be to allow a conferee to 

20 invoke a new "follow me" type of service during a cur- 
rently active conference wherein e.g. the respective 
conferee's connection to the conference could be shift- 
ed/switched from a fixed line in the PSTN (e.g. at a re- 
spective office or residence) to a mobile radio link. Thus, 

25 a conferee could leave home or office to travel else- 
where without disconnecting from the conference and 
without affecting continuity of the conference. 
[0020] Another use, if the PSTN conference centre is 
suitably equipped with speech recognition apparatus 

30 and applications therefor, would be to allow a voice com- 
mand over the PSTN conference connection to be de- 
tected by the speech recognition apparatus and used to 
server to control adjustment of a service condition in ei- 
ther the voice or data path or even in both paths (e.g. 

35 by displaying an indication to all participants stating or 
meaning LETS TAKE A BREAK). 
[0021] Another aspect of this arrangement is that the 
conference server can be configured and connected to 
support inclusion in a conference of "voice-only" and 

40 "data-only" participants, in addition to "voice and data" 
participants (the latter speaking and listening to voice, 
and sending and receiving data). Thus, a "voice-only" 
participant (e.g. a telephone user either without a com- 
puter or without concurrent access to the data network) 

45 could listen and speak to other participants, and a "data- 
only" participant (e.g. with access to the data network 
but either no telephone or no concurrent access to the 
PSTN for voice) could send and receive image data 
without hearing what is said about received data. 

50 [0022] Another aspect is that "voice-only" connec- 
tions could be set up without requiring telephone system 
operator intervention; for instance by having a confer- 
ence master (e.g. the party who originated the respec- 
tive conference) provide prospective voice-only partici- 

55 pants with a pre-arranged password recognizable by 
speech recognition apparatus associated with the con- 
ference switching centre in the PSTN, having each such 
prospective participant call a special number connecting 
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to the speech recognition apparatus, and having the 
speech recognition apparatus and an associated soft- 
ware application transmit appropriate signals to confer- 
ence management facilities of the associated switching 
centre, upon detection of the spoken password, to have 
the latter establish an appropriate connection adding the 
caller to a currently active conference. The associated 
software application also could be used to signal the 
conference server to modify the currently displayed im- 
age so as to indicate addition of another conferee, and 
if desired the new conferee's name. 
[0023] Another aspect of this arrangement is that it 
supports conferencing relative to a variety of different 
customer facilities; thereby supporting transfer of data 
between conferees using an external data network and 
other conferees having a data terminal linked to the 
PSTN, without access to any external data network, as 
well as transfer of voice between conferees using an ex- 
ternal data network for voice transmission and other 
conferees relying exclusively on the PSTN for that func- 
tion. 

[0024] Another aspect is that this conference server 
arrangement could potentially make two-party multime- 
dia conferences using the server more productive and 
efficient than comparable services using totally sepa- 
rate voice and data connections, and that the cost of 
such support could be small enough to allow for attrac- 
tive pricing thereof to end users. 
[0025] These and other aspects, features, advantag- 
es, benefits, and uses of our invention will be further un- 
derstood by considering the following description and 
claims. 

Brief Description Of The Drawings 
[0026] 

Figure 1 schematically illustrates a prior art system 
for multimedia conferencing wherein data can not 
affect voice handling and voice can not affect data 
handling.. 

Figure 2 is a block diagram illustrating PSTN for 
managing voice connections in the system of Figure 
1. 

Figure 3 schematically illustrates application of this 
invention to the system of Figure 1. 

Figure 4 is a block diagram illustrating PSTN facili- 
ties for the arrangement of Fig. 3. 

Figure 5 is a chart providing an overview of how 
voice and data are handled in the prior art system 
of Figures land 2. 

Figure 6 is a chart providing an overview of how 
voice and data are handled in the system of Figures 
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3 and 4. 

Figure 6a is a chart elaborating on information given 
in Figure 6. 

5 

Figure 7 is a flow diagram illustrating how the inven- 
tion can be used to coordinate distribution and dis- 
play of image data with voice presentations. 

10 Figure 8 is a chart showing how the invention can 
be used by a conferee to alter usage of or conditions 
in the voice path of a conference. 

Figure 9 illustrates how the procedure of figure 8 
15 can be used to interrupt a speaker. 

Figure 1 0 illustrates how a similar procedure can be 
used to modify voice amplitude. 

20 Figure 11 illustrates a preferred embodiment of the 
invention. 

Detailed Description 



[0027] The following is a brief description of terms to 
be used in this description and their intended meanings: 

LAN (Local Area Network) is a data communication 
network, either public or private, serving end users 
within a small area (e.g. within a building or geo- 
graphically concentrated commercial enterprise). 

• WAN (Wide Area Network) is a data communication 
network serving a larger geographic area than a 
LAN. The Internet and World Wide Web (hereafter, 
Web) are "species" of public WAN systems that ef- 
fectively span the world, and serve members of the 
public at large having appropriate hardware, soft- 
ware and communication access thereto. 

• PSTN (Public Switched Telephone Network) is the 
aggregate of all telephone communication resourc- 
es available to members of the public at large; in- 
cluding resources providing local, long distance 
land-based, and wireless/cellular telephone servic- 
es. 

• Server means a communication-oriented computer; 
usually with fast internal clock, large memory, large 
storage capacity, and general capability of sustain- 
ing concurrent data communication with multiple 
end users.. 

• URL (Uniform Resource Locator) usually repre- 
sents the address of a specific document, file or dis- 
playable page image on the Internet or Web. It con- 
sists of a collection of characters identifying the da- 
ta, its origin (e.g. the server containing it) and a spe- 
cific signalling protocol used for its transmission. A 
URL may be used by "browsing" software, at loca- 
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tions of Web end users, to designate a file to be re- 
trieved from a remote server on the Web. 
• Conference call (or session) represents a voice 
connection service provided by the PSTN wherein 
two or more end users are connected so that each 5 
can speak and the others can hear what is said by 
the speaker. Participants in a conference call are 
herein termed participants, attendees, conferring 
parties or conferees. 

2. Explanation of Problem Presently Solved 

[0028] This invention concerns multimedia confer- 
ences wherein conferees concurrently may converse 
and exchange data, the latter including displayable text 
and graphic images. In today's facilities for multimedia 
transmission, data and voice are routed over separate 
paths such that it is difficult if not impossible to have 
presentations of image data coordinated with vocal 
presentations in a coherent manner. 
[0029] As exemplified in Figure 1 , a typical contem- 
porary (prior art) facility of this kind routes voice signals 
exclusively through the PSTN, and data signals exclu- 
sively through a parallel public data network like the 
Web, but without any possibility for interaction between 
the two networks; e.g. to allow for coordination of display 
image presentation with vocal presentation, or to allow 
for data signal communications to alter services current- 
ly effective in the PSTN, etc. 

[0030] The PSTN, shown at 1, has conference call 
managing and administering facilities 2 which oversee 
voice signal connections between conference partici- 
pants but have no tie-in to simultaneous data transmis- 
sions between the participants. In figure 1 , A and B rep- 
resent two (of two or more) end-user stations currently 
having a conference connection on the PSTN. Equip- 
ment used at these stations are indicated respectively 
at 5 and 6. Station A's equipment 5 includes a telephone 
7 and computer/display apparatus 8, and station B's 
equipment 6 includes a telephone 9 and computer/dis- 
play apparatus 10. Telephones 7 and 9 are connected 
through transmission lines 11 in the PSTN and (not 
shown) switches administered and controlled by the 
PSTN. Telephones 7 and 9 are not locally connected to 
respective computers in this illustration, but even if so 
connected neither the PSTN nor the Web could act to 
coordinate or otherwise inter-relate voice and data 
transmissions; i.e. only the end users of these stations 
could act to coordinate data transfers with voice, by 
agreeing on a set of protocols beforehand, and such ac- 
tions could not affect services provided in either net- 
work. 

[0031] To exchange data in the typical arrangement 
of Figure 1 , conferees at stations A and B link to a server 
12. The data is transferred e.g. through modems indi- 
cated at 13 and 14, transmission line paths shown col- 
lectively at 15, and a data communication network such 
as the Web shown at 16. In the illustrative example of 



this figure, server 12 and modem 13 are shown as lo- 
cally accessible to station A and other not-shown sta- 
tions via a LAN 17, and remotely accessible to station 
B via the Web 1 6 and modem 14. Thus, in the illustrated 
example, station A and the not-shown other stations 
could be computer stations within a building or a group 
of geographically proximate buildings (e.g. a building or 
buildings locally serving a business enterprise), and 
server 12 and modem 13 could be shared via LAN 17 
by occupants of the respective building(s). Thus, the oc- 
cupants would receive shared data handling and mo- 
dem conversion services, rather than requiring separate 
data handling and modems at their individual offices or 
sites. In contrast, modem 14 is used only by station B 
and is within a few feet of the computer at that station. 
[0032] In this configuration, modems such as 13 and 
14 are required only for linkage to the applicable data 
network such as the Web. Also, server 1 2 can be located 
anywhere within communication reach of the respective 
set of stations that it serves (i.e. a WAN could be sub- 
stituted for LAN 17 if necessary and/or appropriate). 
Futhermore, it should be understood that server 1 2 may 
be only one of plural servers through which the confer- 
ees exchange data. Although lines 1 5 between modems 
13 and 14 are shown as passing through both the Web 
16 and PSTN 1 , in this example they have no connecta- 
bility to the call management facilities 2 of the PSTN. 
[0033] Referring to Figure 2, the prior art call manage- 
ment facilities 2 include a call management system 20 
(e.g. a computer system or the like) which manages and 
administers conventional switching apparatus 21 and 
conventional bridging apparatus 22 to establish and ter- 
minate conference call connections. Apparatus 22 com- 
prises facilities allowing for voice signals to be amplified, 
summed and distributed among the conferees. 
[0034] In this environment, functions and end user 
services provided by the Web are not affectable by voice 
signals in the PSTN, and conditions in the voice con- 
nection path are not affectable by signals sent through 
the data connection/Web. 

[0035] Figure 5 shows how a conference session is 
established and maintained in the prior art environment 
shown in Figures 1 and 2. As shown at 23, each partic- 
ipant establishes separate linkages to the PSTN and to 
a server such as 1 2 in the data network/Web, the former 
exclusively for voice contact and the latter exclusively 
for data interchange. Therefore, each participant re- 
quires simultaneous access to the PSTN and the data 
network via at least two physically or logically separate 
lines. Typically, in this environment, arrangements for 
the conference are made in advance and may require 
assistance of telephone system operators to link indi- 
vidual callers to the conference and announce their 
presence to those already participating. Usually, the ad- 
vance arrangements define a maximum number of at- 
tendees, and once that number is reached no additional 
participants are permitted (unless and until one or more 
current attendees drop out of the conference). 
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[0036] As suggested at 24, this environment provides 
two concurrent but uncoordinatable conferences, one 
handling speech between the participants via voice con- 
nections within the PSTN, and the other handling move- 
ment of displayable data between them via the Web and 
servers such as 12. Such servers may be administered 
by either the Web or entities that are not part of either 
the Web or the PSTN, and they may have security facil- 
ities restricting external access to them which are costly 
to acquire and maintain. If the participants have facilities 
for collaboratively modifying displayed images, all must 
have corresponding non-standardized software appli- 
cations permitting that collaboration. 
[0037] Since the voice and data connections are sep- 
arate and not inter-linkable, signals in either connection 
path can not affect services performed in the other path. 
Thus, data signals sent over the Web can not be used 
to prompt the PSTN to alter its current handling of the 
conference (e.g. to increase reception volume of a hear- 
ing impaired participant, or modify the connection of a 
participant from e.g. a fixed residential or office phone 
to a mobile radio unit), and voice signals can not be used 
to control or coordinate presentation of image data to 
the participants; e.g. in coordination with a voice pres- 
entation. 

[0038] The present invention seeks to resolve these 
and other problems or shortcomings of existing multi- 
media conferencing arrangements, by incorporating 
conference server facilities into the PSTN which are ef- 
fectively linked to both data network(s) such as the Web 
and the PSTN. These conference servers are commu- 
nication-oriented digital computer systems having com- 
munication links to both the PSTN and public data com- 
munication networks like the Internet, Web, etc., and 
their functions and sites of location are administered by 
the PSTN. The conference servers in general will be 
able to communicate with conference management re- 
sources of the PSTN. They also may be able to commu- 
nicate with speech recognition apparatus and associat- 
ed applications within a conference switching centre of 
the PSTN so as to be able to react to voice commands 
issued by conferees; for example, commands to distrib- 
ute data representing specific display images provided 
by the conferee issuing the command. These servers 
and their integration into the PSTN would enable the 
PSTN to offer cost-effective conference management 
services to its end users that are either completely new 
or presently considered impractical for existing PSTN 
facilities. Such services would include cost effective 
handling of voice conferences over conventional (un- 
leased) lines in the "plain old telephone system" 
(POTS), concurrent with distribution of data over public 
data networks parallel to the PSTN (e.g. the Web), 
wherein conferees would be able to use their data net- 
work connections to control conditions of operation in 
the voice connection path, and conversely able to use 
voice commands to control functions performed in the 
data network, for example, to control distribution of im- 
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age data in coordination with voice presentations. Thus, 
a conferee would be able to administer and/or invoke 
specific services in either the PSTN or the parallel data 
network via signals sent on the other network. An ex- 

5 ample would be that a conferee could issue a "follow 
me" data request to switch their current voice connec- 
tion (e.g. from a stationary phone to a mobile radio one), 
or another data request to have the amplitude or other 
parameters of voice transmitted to them changed (e.g. 

10 to adjust the sound delivered to a hearing impaired con- 
feree). 

[0039] Although conference calls on the PSTN here- 
tofore usually have involved more than two participants, 
with our inventive arrangement a conference call may 

15 be used as an efficient method for two parties to con- 
verse while concurrently exchanging data (the term "ef- 
ficient method" is used here to imply a process that 
could be more cost-effective and productive than a sim- 
ple voice connection over the PSTN concurrent with a 

20 simple and completely separate data connection over 
the Web or another line path within the PSTN). 
[0040] Those skilled in the art will recognize that the 
arrangement of Figures 1 , 2 and 5 is not the only previ- 
ously known technique for sustaining concurrent multi- 

25 media conferences. For instance, presently known soft- 
ware applications for the Internet/Web allow users of 
those networks and respective applications to effective- 
ly carry on "long distance" voice conversations using on- 
ly local resources of the PSTN to link the users to locally 

30 accessible servers. These applications however have 
generally tended to provide voice transfer qualities infe- 
rior to (albeit cheaper than) equivalent long distance 
voice connections on the PSTN; generally due to the 
limited bandwidth available within public data networks 

35 today for such communications, and indefinite routing 
latencies (delays) of those networks. 
[0041] There are also applications purporting to pro- 
vide integrated handling of voice and data over data net- 
works like the Web, as well as conferencing services in 

40 respect to such handling. But they too have limitations 
in respect to quality due mainly to bandwidth limitations 
deriving from the nature of today's network infrastruc- 
tures and from the integrated handling of voice and data 
through common transmission paths. Furthermore, 

45 such arrangements are incapable of enabling their us- 
ers to efficiently coordinate distribution of displayable 
data with voice presentations of respective users. 
[0042] It is also possible for users to converse and 
transfer data concurrently over leased lines, but such 

so lines would be considerably more costly than the type 
of conference services presently contemplated. 
[0043] The present invention seeks to provide cost- 
effective conferencing of voice and data such that: 1 ) 
neither process is degraded by bandwidth limitations; 2) 

55 transmission of data between conferees can be coordi- 
nated efficiently with voice remarks of the conferees; 
and 3) services within either the PSTN or a parallel data 
network such as the Web can be efficiently administered 
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and/or invoked by conferees during a conference. 

3. The Present Solution 

[0044] An overview of how the present invention 
solves the foregoing problems is presented in figures 3, 

4, 6 and 6a. 

[0045] Figure 3 shows the arrangement of Figure 1 
modified in accordance with our invention. Elements 
functionally identical to those in Figure 1 are denoted 
with identical numbers. Consequently this shows that 
novel elements of the invention are in the conference 
control centre 2a and are associated at least in part with 
the data path represented by lines 28, 28-a. 
[0046] Figure 4 shows that conference centre 2a in- 
cludes a conference management system 20a and a 
conference server system 30. Conference server 30 is 
a communications adapted computer system main- 
tained and administered by the PSTN. It's loose integra- 
tion with call management resources of the PSTN, in- 
cluding its connection 31 to conference management fa- 
cilities such as 20a, enable it to control services per- 
formed relative to an existing PSTN conference connec- 
tion, in response to signals sent over a path external to 
the PSTN; exemplified by the indicated external connec- 
tion^) 32 to the Web (these connections being viewable 
as extending to line 28-a in Figure 3). 
[0047] These external connections permit new con- 
ferees to easily be added to an active conference with- 
out assistance of telephone company operators. Such 
added conferees may be pre-scheduled to join the con- 
ference at its inception, or they may join on an ad hoc 
basis, and their additions may be displayed to prior par- 
ticipants eliminating need for announcements by assist- 
ing telephone operators. They also enable conferees to 
easily and economically control other PSTN confer- 
ence-useful services (either before or during a confer- 
ence) without operator assistance; e.g. to vary connec- 
tions of participants moving between fixed desk and mo- 
bile cellular telephones (in a manner further discussed 
later on). 

[0048] Conference management system 20a differs 
from "prior art" conference management system 20 of 
Figure 2 primarily in its connection 31 to conference 
server 30, and in functions and services derivative from 
that connection. As discussed later, speech recognition 
apparatus and associated applications within the PSTN 
may be linked to conference bridge 22, for recognizing 
voice commands of conference participants and ena- 
bling either management system 20a or the data net- 
work, or both, to perform actions associated with such 
commands; e.g. real time actions instantly affecting con- 
ditions in either the voice connection path, the data 
transmission path, or both. 

[0049] It will also be shown later that although partic- 
ipating stations in the arrangements of Figures 1 and 3 
have separate telephones and computers, as well as 
separate communication paths for each, the conference 



centre arrangement of Figure 4 would easily be adapted 
to serve conferees having only a telephone, or only a 
computer, or conferees having both a telephone and 
computer operating through a single physical line con- 

5 necting to either the PSTN or the Web. Adjustments 
needed to support these reduced configurations are dis- 
cussed later in this description. Furthermore, although 
station equipment external to the PSTN and Web is 
shown in Figure 3 as identical to that in Figure 1, it 

10 should be understood that this is done only to facilitate 
comparison of the present inventive solution to prior art. 
Therefore, it should be understood, and will become ap- 
parent from later parts of this description, that stations 
having many different forms of equipment are support- 

15 able by the present arrangement. 

[0050] Figure 6 provides an overview of capabilities 
afforded by the present conference server arrangement. 
[0051] As indicated at 36, conference participants in 
both voice and data aspects of a conference still estab- 

20 nsh parallel voice and data connections over physically 
or logically separate lines extending to the PSTN and 
data network. However, as indicated at 37, data trans- 
ferred between the conferees is funnelled through con- 
ference server 30. Consequently, servers external to the 

25 PSTN may not be required for presently contemplated 
conference services, except perhaps for providing data 
collection, storage and distribution services within e.g. 
private enterprises (office buildings, plants, etc.). In oth- 
er words, end users at self-contained participating sta- 

30 tions like station B, having unshared data handling and 
modem facilities, need only link to PSTN conference 
servers such as 30 and need not have any direct links 
to other servers such as 12. 

[0052] As suggested at 37, since server 30 has direct 

35 links to conference management facilities of the PSTN, 
image data transmissions on the Web are coordinatable 
with vocal aspects of the conference, and other data 
transmissions may be used by participants to alter or 
invoke services affecting their voice connections (e.g. 

40 volume of voice signals sent to them, interruption of 
speakers, switching phone connections e.g. from fixed 
lines to mobile radio links, etc.). Block 37 also indicates 
that data security may be provided by the conference 
server at "relatively small cost"; since that server and its 

45 functions are effectively shared by all users of the re- 
spective PSTN conference management and bridging 
facilities, rather than by potentially fewer users of (pos- 
sibly a larger number of) servers such as 12, whence its 
security functions may be apportioned among a poten- 

50 tially larger population of users than that of servers such 
as 12 external to the PSTN. This should be more fully 
understood from the description below of Figure 11. 
[0053] Figure 6a elaborates on functions indicated in 
Figure 6. Voice signals flow only through switch and con- 

55 ference bridge connections in the PSTN (block 40). Data 
signals flow only through data connection paths be- 
tween the conference server and conferees, generally 
through the Web, and generally over connection paths 
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separate from the voice connection (block 41). Voice 
commands/requests may be detected by speech recog- 
nition apparatus in the PSTN, and used in associated 
control applications to initiate controlling actions by ei- 
ther the conference manager, the conference server or 5 
both (block 42); i.e. actions affecting conditions in either 
the voice path, the data path or both. Data passed 
through the data network includes data representing 
displayable images, and conferee requests and other 
control signals such as coordination cues (block 43). 
Cues may be signalled to the conference server by a 
conferee and handled by the conference server to ap- 
pear as displayed icons or other viewable features at 
stations of other conferees (also block 43). Requests 
may be forwarded to the conference manager for han- 
dling if the associated functions are not directly control- 
lable by the conference server, such as functions requir- 
ing changes of conditions in voice connection circuits 
(block 43). 

[0054] It should be noted here that this invention con- 
templates that requests for changes in voice connection 
circuits may be sent to the conference server either over 
a data network such as the web or a third parallel com- 
munication network, such as a network handling two 
way pager communications for conferees equipped with 
two way pager units. 

[0055] Figure 7 illustrates how data representing dis- 
playable images (e.g. charts) may be presented to all 
conferees in coordination with a voice presentation by 
one of the conferees. The image data is sent by a con- 
ference master or moderator (e.g. a party originating the 
conference, a party owning or controlling the data, a par- 
ty scheduled to make the voice presentation) to the con- 
ference server, either before or during the conference 
and stored by the conference server (block 50). There- 
after the data is distributed by the conference server, 
one page at a time in response to coordinating cues giv- 
en by the voice presenter (block 51). Images stored in 
the conference server may be distributed to the confer- 
ees one at a time, for immediate display upon reception, 
and each page may be sent in response to a cue from 
the master/moderator. This cue may be manifested ei- 
ther as data representing a request to have the confer- 
ence server distribute a specified chart (or a chart next 
in a predetermined sequence, etc.), or as an equivalent 
voice command picked by speech recognition appara- 
tus linked to the voice connection of the conference and 
transferred to the conference server. 
[0056] Figure 7 indicates further that conferees hav- 
ing collaboration software (software enabling them to 
collaboratively edit a displayed image) may make edit- 
ing changes to a currently viewed page and have the 
changes communicated to the other conferees via the 
conference server (block 52). Such changes may be 
communicated in any of several ways, either as trans- 
missions indicating only the edited changes, or as trans- 
missions including data representing the whole page 
with the edited changes, or in any other manner consist- 



ent with the intended communication effect. 
[0057] Collaborative editing may be achieved in sev- 
eral ways. The method preferred presently would in- 
volve use at participating stations of browser software 
capable of receiving and executing mini-programs or 
"applets" suited to tasks required for performing collab- 
orative modifications. Such applets may for instance be 
written in a commonly used language such as the 
Java™ language (Java is a Trademark of Sun Microsys- 
tems company), which is interpretable by a number of 
existing browsers constructed to interpret Java scripts 
(e.g. Netscape Navigator, Microsoft Explorer, etc), 
stored in the conference server, and transferred from the 
conference server to conference participants as the lat- 
ter sign onto the data network for a conference session. 
These applets could be constructed to either self-de- 
struct/erase at participant stations when a conference 
ends, or to remain stored at participant stations after a 
conference for use in future conferences (since the serv- 
er would not easily distinguish which stations have pre- 
viously transferred applets, the preferred method would 
be to distribute them as a conference begins and have 
them self-destruct at participant stations when the con- 
ference ends. Such applets could enable conferees to 
modify graphic and/or text elements of a currently dis- 
played page image, and communicate modifications to 
the conference server for distribution to stations of other 
conferees. 

[0058] A less preferred method would be to require all 
participants to have particular proprietary software (e.g. 
Lotus Notes) enabling them to create, edit and inter- 
change displayable documents within a special commu- 
nication environment required by that software. 
[0059] As shown at 53, conferees requiring special 
services during a conference, that are invocable through 
the data connection, send data representing appropri- 
ate requests to the conference server, and the latter re- 
acts appropriately to have the requested function per- 
formed if it is available. Box 53 also shows that confer- 
ees may send "other data" (data other than edited im- 
ages originated by the conference master and requests 
for service changes) and that such other data may be 
distributed to the other conferees if appropriate. 
[0060] Figure 8 illustrates an example of one such 
special service request for action pertinent to the voice 
conference. As shown at 55, data representing the re- 
quest is sent to the conference server. As suggested at 
56, the server determines if the requested service is 
available, and either takes action suited to the request 
directly or signals the conference manager to do so, de- 
pending upon where the requested action is executable. 
As seen at 57, the server then returns to the requestor 
either an acknowledgement of the request or an indica- 
tion that the requested service is unavailable. 
[0061] Figure 9 gives an example of a data request 
pertaining to conditions in the voice circuits. In this ex- 
ample, the requestor seeks to interrupt a party currently 
speaking. At 60, the request is sent to the conference 
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server, and at 61 the server responds by illuminating an 
icon on the image currently displayed to the participants; 
e.g. an icon representing ASK QUESTION, LET ME 
TALK, etc. This may be done either by signalling brows- 
ers at participant stations to modify the portion of the 
displayed image containing the icon, or by signalling that 
the current image is "stale" and requires reloading (in 
which case the browsers would request reloads and re- 
ceive the current image with the icon illuminated), or by 
any other action resulting in an appropriate visual cue 
to the speaker that another party wants to speak. 
[0062] Figure 1 0 illustrates another example of action 
pertaining to the voice conference. Here, the request 
(operation 64) is to alter a characteristic (e.g. volume) 
of voice signals that are being sent to the requestor (e. 
g. a hearing impaired party). In this case, the conference 
server forwards a corresponding request to the confer- 
ence manager (operation 65), and the latter takes action 
66 relative to the voice path (conference bridge, etc.) to 
effect the desired change. A similar request action could 
be used to have the conference server and conference 
manager change the phone line connection of a re- 
questing participant; e.g. to switch that participant's link 
to the conference from a fixed line (residential or office) 
to a mobile radio link. In this last example, the switching 
action could be implemented by having the conference 
manager first communicate with a mobile switching of- 
fice to establish the mobile radio link to the conference 
bridge, and then act to sever the connection of the re- 
questor's fixed line. 

[0063] Figure 11 schematically illustrates a confer- 
ence control system, in accordance with the present in- 
vention, which shows the different configurations of user 
stations that are supportable and how such support is 
given. As such, this may be considered as showing the 
best mode contemplated for carrying out our invention. 
[0064] This figure shows four user stations - stations 
A, B, C and D - having various different configurations 
of equipment which for the following discussion are as- 
sumed to be actively connected, via conferencing centre 
70 in the PSTN, in a multimedia conferencing session 
(i.e. users of these stations are participants in that ses- 
sion). 

[0065] Station A contains only a single analog tele- 
phone 72; i.e. its user is a voice-only participant in the 
conference. Station C contains an analog telephone 74, 
digital computer 76 and modem 78, with the telephone 
connecting to conference centre 70 in a first connection 
path within the PSTN that is described below, and the 
computer and modem connecting in a physically sepa- 
rate second path within the PSTN, also described be- 
low; i.e. its user is a participant in both voice and data 
elements of the conference, whose voice and data flow 
through physically different paths within the PSTN. Sta- 
tion D contains only a computer 80, modem 81 and 
speaker 82 and microphone 84 both attached to the 
computer; i.e. its user is a voice and data participant 
whose voice and data are constrained to pass through 



the respective computer 80 and modem 81 , and there- 
fore have only a single physical path of connection to 
the conference centre (which path, in the illustrated ar- 
rangement, is entirely within the PSTN). Finally, station 

5 B contains an analog telephone 85 and digital computer 
88 which connect to the conference centre through 
physically separate paths described below, one in the 
PSTN (the voice path) and the other external to the 
PSTN (the data path). 

10 [0066] Computers at these stations (those other than 
station A) are shown as laptop style personal comput- 
ers, but it is understood that they may be computers of 
any physical form (e.g. laptop, desktop, floor console, 
or even computers integral to an appliance such as a 

15 TV monitor), and need not even be personal type com- 
puters (i.e. they could even be computers like main- 
frames or such that are larger in size and operating ca- 
pabilities than personal computers). The only require- 
ment is that they contain browser software that is com- 

20 patible with applets or scripts written in a common lan- 
guage such as Java, and that communicate with proto- 
cols that are compatible with those employed at the con- 
ference server to be described below. 
[0067] Conference centre 70 comprises call connect- 

25 jng circuits 90 containing switching circuits needed for 
simple non-conference connections as well as bridging 
circuits required for conference connections. The size 
and capabilities of this element are dependent upon the 
anticipated peak traffic in non-conference and confer- 

30 ence calls for the respective conference centre. 

[0068] Conference centre 70 also comprises a call 
management facility 92, a conference server facility 93, 
an administrative facility 94, an (optional) speech rec- 
ognition facility 95 (for speech recognition applications 

35 including those relevant to conference management), 
and an (optional) archive facility 96 (for voice and data 
archiving applications that could e.g. allow voice and da- 
ta portions of a conference to be replayed to conference 
participants at some later date). Facilities 92, 93, 95 and 

40 96 have direct connections to connecting circuits 90 for 
handling signals to and from end user stations. All of the 
facilities 92-96 connect to a LAN 1 00 for communicating 
with each other. The connection circuits 90, facilities 
92-96 and LAN 100 are all components of conference 

45 cents 70, and the latter is a component of the PSTN. 
[0069] Telephone 72 in station A connects to connect- 
ing circuits\90 through a part of the PSTN indicated at 
104. Telephone 74 in station C connects Xo circuits 90 
through a part of the PSTN shown at 106, while compu- 

50 ter 76 and modem 78 in the same station connect to 
conference server 93 through another part of the PSTN 

108 and a modem bank 109 (i.e. this station has two 
physically separate lines locally connecting to the PSTN 
and having voice routed through the switch circuits 90 

55 and data routed to the conference server over a path 
parallel to and separate from circuits 90). The computer 
80 and modem 81 in station D connect to modem bank 

109 via a link to PSTN portion 1 08 that is physically sep- 
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arate from that of modem 78 in station C. Consequently, 
the voice traffic associated with speaker and micro- 
phone attachments 82 and 83 in station D flows through 
computer 80, modem 81, PSTN portion 108, and mo- 
dem bank 109, interspersed with any other data issued 5 
by or directed to computer 80; and all vojce and data 
signals routed between modem 81 and PSTN portion 
108 are in a common (analog) form. 
[0070] Modem bank 1 09 serves to convert signals be- 
tween analog and digital forms. Signals sent by modems 
78 and 81 are converted to (baseband) digital form by 
modem bank 109 and passed to conference server 93 
in the latter form. Signals sent by conference server 93 
to modems 78 and 81 (and their respective computers) 
are converted in bank 109 from (baseband) digital form 
to analog form. 

[0071] Telephone 85 in station B connects to circuits 
90 via portion 110 of the PSTN, while computer 88 in 
the same station connects to conference centre 70 
through a path 112 external to the PSTN. Path 112 com- 
prises a LAN 114 connecting computer 88 and other 
(not-shown) computers to conference server 93 through 
a gateway/firewall security element 115 and a portion 
116 of the Web (or other data network). 
[0072] Notice that the connections between confer- 
ence server 93 and computers 78 (station C) and 80 
(station D) are entirely within the PSTN (through modem 
bank 109, which is a component of conference centre 
70 and PSTN portion 1 08), whereas the connection be- 
tween the server and computer 88 is external to the 
PSTN (i.e. through Web portion 116, and private net- 
work 1 14). Notice also that signals sent between modem 
bank 109 and computers 78 and 80 are analog in form, 
whereas those sent between server 93 and the modem 
bank are in digital form and those sent between server 
93 and computer 88 are in analog form in transit through 
the Web and in baseband digital form in transit through 
LAN 114 and firewall 115. Signals passing between fire- 
wall 115 and the Web are converted between digital and 
analog forms through not-shown modems and other cir- 
cuits located at the access node of the Web at which 
signals from the firewall pass into the Web and signals 
leaving the Web transfer to the firewall. 
[0073] An optional direct connection between modem 
81 and web portion 116 is suggested at 117, to indicate 
that the present conference server arrangement can be 
used to accommodate voice and data transmission 
through the web, and link voice portions thereof to voice 
conference facilities within the PSTN. 
[0074] Conference server 93 has a voice only connec- 
tion (V) to switch circuits 90 and a voice and data con- 
nection (V/D) to the same circuits. These connections 
are required for routing some of the conference voice 
and data traffic between the participants, where the re- 
spective traffic signals are required to pass between 
connection circuits 90 and either modem bank 109 or 
the parallel data network 114-116 linked to station B. 
The voice only connection is used e.g. for routing analog 
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voice signals through the server, between telephones 
72 (station A), 74 (station C) and 85 (station B). The 
voice and data connection is used e.g. for routing analog 
voice signals through the server, between the speaker 
and microphone attachments to computer 80 at station 
D and any of the telephones at stations A, C and D, as 
well as for routing analog data signals exclusive of voice 
between computer 80 and the other station computers 
76 (station C) and 88 (station B). 
[0075] It should be noted that interspersed voice and 
data signals passed from station D to the conference 
server, via modem bank 1 09, usually will have to be sep- 
arated at the server 93 to permit the voice signals to be 
transferred via circuits 90 to the telephones at the other 
stations, while the data is subjected to other handling 
(for example, storage in the server, or interpretation by 
the server as commands, cues or requests, etc.). Simi- 
larly, voice and data transferred to station D would have 
to flow through the modem bank and would have to be 
appropriately merged into a common form suitable for 
handling by modem 81 at station D. These separation 
and merger functions are state of the art logical func- 
tions that can be performed either in the modem bank 
by suitable logic, or by the server 93 if the data is in a 
form suitable for reading by that entity. 
[0076] It should be understood that in conducting con- 
ference voice and/or data signals from conference con- 
nection circuits in the PSTN to external networks such 
as the Web, the present conference server would re- 
ceive the respective signals along with an indication 
identifying the associated conference and route the re- 
spective signals to appropriate external paths applica- 
ble to the associated conference. Conversely, in con- 
ducting voice and/or data signals from an external 
source to the conference connection circuits in the 
PSTN, the conference server would receive the respec- 
tive signals, recognize the conference associated with 
their origin, and transfer the signals to the PSTN con- 
nection circuits along with an identification of the asso- 
ciated conference, causing the signals to be routed over 
telephone company circuits and lines to destinations ap- 
propriate for that conference. 

[0077] It should be recognized that the foregoing ar- 
rangement would be readily adaptable to perform any 
or all of the following functions: 

• add conferees to an existing conference (either pre- 
determined individuals or others entering on an ad 
hoc basis) without telephone operator assistance; 

• announce entry of new conferees via computer dis- 
plays, also without operator assistance 

• drop conferees from an existing conference without 
operator assistance; 

• split a conference into separate sub-conferences 
involving discrete sub-groups of the group partici- 
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pating in the parent conference; 

• selectively block portions of a conference (voice 
and/or data) from presentation to some partici- 
pants; e.g. to present confidential data to partici- 
pants authorized to receive such and block it from 
participants not authorized to receive it. 

[0078] The foregoing and other aspects of the inven- 
tion will be further appreciated from the following claims. 



Claims 

1. A voice and data conferencing system comprising: 

a conference server (30) communicating with a 
conference management system (20a). said 
conference server having controlling connec- 
tions (31, 32) to both conferencing circuits and 
a data communication network; said controlling 
connections (32) enabling said conference 
server to direct transfers of data signals 
through said data communications network, 
said data being transferred including data rep- 
resenting displayable image data and data rep- 
resenting functional requests and commands 
issued by individual conference participants 
and requiring attention by said conference 
management system; 

characterised in that: 

the voice and data conferencing system is for 
the public switched telephone network (PSTN), 
said PSTN (1 ) having said conferencing circuits 
(2a) entirely internal thereto for routing analog 
voice signals between end users (5,6) of said 
PSTN, and also having said conference man- 
agement system (20a) entirely internal thereto 
for managing operations in said conferencing 
circuits to provide conference connections be- 
tween said end users; 

the conference server operating as an internal 
part of said PSTN; 

the data communication network (16) is exter- 
nal to said PSTN; 

said controlling connections enable said con- 
ference server to direct transfers of data signals 
through said external data communication net- 
work, in coordination with a voice conference 
established in said internal conferencing cir- 
cuits; said data being transferred in parallel 
through said external data network, between 
participants of said established voice confer- 



ence having connections to said external net- 
work, in coordination with voice signals trans- 
mitted between said participants through said 
internal conferencing circuits; said functional 

5 requests including data transfer requests, rout- 

ed from a said participant to said conference 
server, to have said conference server manage 
distribution of data representing said displaya- 
ble image data, said image data having been 

10 previously furnished to said conference server 

by a participant issuing the request, to other 
said participants; 

said controlling connections and requests be- 
15 ing used to have said conference management 

system distribute said data representing said 
displayable image data between said confer- 
ence participants in coordination with a vocal 
presentation being made by a requesting par- 
20 ticipant; said controlling connections, requests, 

and commands further being used for having 
said conference management system perform 
controlling functions relative to both said inter- 
nal conferencing circuits and said external data 
25 network. 

2. A voice and data conferencing system in accord- 
ance with claim 1 wherein said means controlling 
connections (31) between said conference server 

30 (30) and both said conferencing circuits (21) and 
conference management system (20a) are used, in 
response to certain said functional requests, to sig- 
nal said conference management system to vary 
conditions of operation in said conferencing circuits. 

35 

3. A conferencing system in accordance with claim 1 
or claim 2 wherein said conference server (30) fur- 
ther comprises: 

40 means responsive to signals received from a 

speech-recognition means for distributing indi- 
vidual pages of said displayable image data to 
said participants and for performing other func- 
tions relevant to said voice conference of said 

45 participants. 

4. A conferencing system in accordance with claim 2 
wherein said signals to said conference manage- 
ment system (20a) to vary conditions in said con- 
so ferencing circuits (21 ) are used to cause the tele- 
phone connection of a said conference participant 
(5,6) to be changed so as to enable the respective 
participant to change between wired and wireless 
channels of connection to said PSTN (1 ) without in- 

55 terrupting continuity of a currently active confer- 
ence. 

5. A conferencing system in accordance with claim 2 
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wherein said signals to said conference manage- 
ment system (20a) to vary conditions in said con- 
ferencing circuits (21) are used to vary the volume 
of sound currently being delivered to a telephone 
receiver used by one of said participants. 

6. A conferencing system in accordance with any pre- 
ceding claim wherein said controlling connections 
(31) between said conference server (30) and said 
conferencing circuits (20a) are useful to enable said 
conference server to transfer signals representing 
voice and displayable image information between 
said internal conferencing circuits and voice and 
displayable image handling equipment connected 
to said external data network (16), and also to en- 
able said conference server to transfer signals rep- 
resenting displayable image data between said dis- 
playable image handling equipments connected to 
said external data network; thereby enabling PSTN 
(1 ) end users lacking connections to said data net- 
work to send and receive data relative to the data 
network via the PSTN. 

7. A conferencing system in accordance with any pre- 
ceding claim wherein said external data communi- 
cations network (16), which does not form part of 
the conferencing system and to which the confer- 
ence server (30) has a controlling connection, is a 
public data network that is accessible to any mem- 
ber of the public complying with requirements of the 
respective network. 

8. A conferencing system in accordance with any pre- 
ceding claim wherein said conference server (30) 
is effective to form a circuit path between said inter- 
nal conferencing circuits (20a) in said PSTN (1 ) and 
said external data network (16) for selectively rout- 
ing signals representing both voice and data be- 
tween said conferencing circuits and said external 
data network. 

9. A conferencing system in accordance with claim 8 
wherein said circuit path formed by said conference 
server (30) enables said PSTN (1 ) and external da- 
ta network (16) to support conferences between 
conference participants having different equipment 
and different communication connections including 
a participant having a conventional telephone (74, 
85) linked to the PSTN (106, 110) and a computer 
(76, 88) separately linked to said external data net- 
work (1 1 6); a participant having only a conventional 
telephone (72) linked to the PSTN (104); a partici- 
pant having only a computer linked to said external 
data network; said computer unequipped for han- 
dling voice; a participant having only a computer 
(80) linked to said external data network (116); said 
computer being equipped for handling both voice 
and data transmissions relative to the respective 



participant; a participant having only a device linked 
to said external network; said device being 
equipped only for handling voice communications 
relative to the respective participant; or a participant 
5 having only a computer linked to said PSTN; said 
computer being equipped for handling both voice 
and data transmissions relative to said PSTN. 

10. A conferencing system in accordance with any pre- 
10 ceding claim wherein said conference server (30) 
includes: 

means linking said conference server to a mo- 
dem bank (109) for routing voice and data sig- 
15 nals between said internal conferencing circuits 

(20a) and said modem bank and between said 
external data network (116)and said modem 
bank. 

20 1 1 . A conferencing system in accordance with claim 1 0 
wherein said conference server (30) is effective to: 

separate voice and data signals received in 
said modem bank (109) from a single said par- 

25 ticipant; 

route the separated voice signals to said con- 
ferencing circuits (20a) for connection to tele- 
phones of all conference participants having 
telephones; and 

30 selectively route some of the separated data 

signals to said conferencing circuits, for deliv- 
ery to participants having computers directly 
connected to said PSTN, and route the remain- 
ing separated data signals to said external data 

35 network for delivery to participants having com- 

puters directly connected to said external data 
network. 



40 Patentanspruche 

1. Sprach- und Daten konferenzsystem, das Folgen- 
des umfasst: 

45 einen Konferenzserver (30), der mit einem 

Konferenzverwaltungssystem (20a) kommuni- 
ziert, wobei der Konferenzserver Steuerverbin- 
dungen (31, 32) zu sowohl den Konferenz- 
schaltungen als auch einem DatenObertra- 

50 gungsnetzwerk hat; wobei die Steuerverbin- 

dungen (32) dem Konferenzserver ermogli- 
chen, Obertragungen von Datensignalen durch 
das Obertragungsnetzwerk zu steuern, wobei 
die Daten, die ubertragen werden, Daten ent- 

55 halten, die anzeigbare Bilddaten darstellen, so- 

wie Daten, die Funktionsanforderungen und 
Befehle darstellen, die von einzelnen Konfe- 
renzteilnehmern ausgegeben werden und die 
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Aufmerksamkeit des Konferenzverwaltungssy- 
stems erfordern; 

dadurch gekennzeichnet, dass: 

5 

das Sprach- und Datenkonferenzsystem fDr 
das offentliche TelefonwShlnetz (PSTN) be- 
stimmt ist, wobei sich die Konferenzschaltun- 
gen (2a) vollkommen innerhalb des PSTN (1 ) 
befinden, um analoge Sprachsignale zwischen 10 
Endbenutzern (5, 6) des PSTN weiterzuleiten, 
und wobei sich auch das Konferenzverwal- 
tungssystem (20a) vollkommen innerhalb des 
PSTN befindet, um Operationen in den Konfe- 
renzschaltungen zu verwalten, um zwischen *5 
den Endbenutzern Konferenzverbindungen 
bereitzustellen; 

der Konferenzserver als ein interner Teil des 
PSTN arbeitet; 20 

sich das Datenubertragungsnetzwerk (16) au- 
lierhalb des PSTN befindet; 

die Steuerverbindungen dem Konferenzserver 25 
ermdglichen, Obertragungen von Datensigna- 
len durch das externe Datenubertragungsnetz- 
werk abgestimmt auf eine Sprachkonferenz zu 
steuern, die in den intemen Konferenzschal- 
tungen aufgebaut wird; wobei die Daten paral- 30 
lei durch das externe Datennetzwerk zwischen 
Teilnehmern der aufgebauten Sprachkonfe- 
renz, die Verbindungen zu dem externen Netz- 
werk haben, abgestimmt auf Sprachsignale 
ubertragen werden, die zwischen den Teilneh- 35 
mern durch die internen Konferenzschaltungen 
ubertragen werden; wobei zu den Funktionsan- 
forderungen Datenubertragungsanforderun- 
gen gehoren, die von einem Teilnehmer an den 
Konferenzserver geleitet werden, damit der 40 
Konferenzserver die Verteilung von Daten, die 
die anzeigbaren Bilddaten darstellen, an ande- 
re der Teilnehmer handhabt, wobei die Bildda- 
ten dem Konferenzserver zuvor von einem Teil- 
nehmer bereitgestellt wurden, der die Anforde- *5 
rung ausgegeben hat; 

die Steuerverbindungen und Anforderungen 
verwendet werden, um das Konferenzverwal- 
tungssystem zu veranlassen, die Daten , die die so 
anzeigbaren Bilddaten darstellen, zwischen 
den Konferenzteilnehmern abgestimmt auf ei- 
ne Sprach presentation, die von einem anfor- 
dernden Teilnehmer gemacht wird, zu vertei- 
len; wobei die Steuerverbindungen, Anforde- 55 
rungen und Befehle des Weiteren dazu dienen, 
das Konferenzverwaltungssystem dazu zu ver- 
anlassen, Steuerfunktionen in Bezug auf die in- 



temen Konferenzschaltungen und auch das 
externe Datennetzwerk auszufQhren. 

2. Sprach- und Datenkonferenzsystem nach An- 
spruch 1 , wobei die Mittel, die Verbindungen (31 ) 
zwischen dem Konferenzserver (30) und sowohl 
den Konferenzschaltungen (21 ) als auch dem Kon- 
ferenzverwaltungssystem (20a) steuern, dazu ver- 
wendet werden, dem Konferenzverwaltungssystem 
als Antwort auf bestimmte der Funktionsanforde- 
rungen zu signalisieren, die Betriebsbedingungen 
in den Konferenzschaltungen zu andern. 

3. Konferenzsystem nach Anspruch 1 Oder Anspruch 
2, wobei der Konferenzserver (30) des Weiteren 
Folgendes umfasst: 

Mittel, die auf von einem Spracherkennungs- 
mittel empfangene Signale ansprechen, um 
einzelne Seiten der anzeigbaren Bilddaten an 
die Teilnehmer zu verteilen und um andere 
Funktion auszufQhren, die fOr die Sprachkonfe- 
renz der Teilnehmer wichtig sind. 

4. Konferenzsystem nach Anspruch 2, wobei die Si- 
gnale an das Konferenzverwaltungssystem (20a) 
zur Anderung der Bedingungen in den Konferenz- 
schaltungen (21) dazu dienen, die Anderung der 
Telefonverbindung eines Konferenzteilnehmers (5, 
6) zu veranlassen, um dem jeweiligen Teilnehmer 
zu ermoglichen, zwischen drahtgebundenen und 
drahtlosen Verbindungskanalen zu dem PSTN (1 ) 
zu wechseln, ohne den Fortgang einer gerade ak- 
tiven Konferenz zu unterbrechen. 

5. Konferenzsystem nach Anspruch 2, wobei die Si- 
gnale an das Konferenzverwaltungssystem (20a) 
zur Anderung der Bedingungen in den Konferenz- 
schaltungen (21) dazu dienen, die Lautstarke zu 
andern, die gerade einem Telefonempfanger bereit- 
gestellt wird, der von einem der Teilnehmer verwen- 
det wird. 

6. Konferenzsystem nach einem vorhergehenden An- 
spruch, wobei die Steuerverbindungen (31) zwi- 
schen dem Konferenzserver (30) und den Konfe- 
renzschaltungen (20a) dazu dienen, dem Konfe- 
renzserver die Obertragung von Signalen, die Spra- 
che und anzeigbare Bildinformationen darstellen, 
zwischen den internen Konferenzschaltungen und 
der Vprrichtung, die Sprache und anzeigbare Bilder 
handhabt, welche mitdem externen Datennetzwerk 
(16) verbunden ist, und auch die Obertragung von 
Signalen, die anzeigbare Bilddaten darstellen, zwi- 
schen den Vorrichtungen zur Handhabung von an- 
zeigbaren Bildem, welche mit dem externen Daten- 
netzwerk verbunden sind, zu ermSglichen, wodurch 
Endbenutzer des PSTN (1), die keine Verbindun- 
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gen zu dem Datennetzwerk haben, Daten in Bezug 
auf das Datennetzwerk Ober das PSTN senden und 
empfangen kbnnen. 

7. Konferenzsystem nach einem vorhergehenden An- 
spruch, wobei das externe DatenQbertragungs- 
netzwerk (16), das nichtTeil des Konferenzsystems 
ist und zu dem der Konferenzserver (30) eine Steu- 
erverbindung hat, ein Offentliches Datennetzwerk 
ist, das jedem Teil der Offentlichkeit, der die Anfor- 
derungen des entsprechenden Netzwerks erfOllt, 
zugSnglich ist. 

8. Konferenzsystem nach einem vorhergehenden An- 
spruch, wobei der Konferenzserver (30) dazu dient, 
einen Schaltungspfad zwischen den internen Kon- 
ferenzschaltungen (20a) in dem PSTN (1 ) und dem 
externen Datennetzwerk (16) zu bilden, urn Signa- 
le, die sowohl Sprache als auch Daten darstellen, 
zwischen den Konferenzschaltungen und dem ex- 
ternen Datennetzwerk selektiv weiterzuleiten. 

9. Konferenzsystem nach Anspruch 8, wobei der von 
dem Konferenzserver (30) gebildete Schaltungs- 
pfad dem PSTN (1) und dem externen Datennetz- 
werk (16) die UnterstOtzung von Konferenzen zwi- 
schen Konferenzteilnehmern, dieuberunterschied- 
liche Ausrustungen und unterschiedliche Obertra- 
gungsverbindungen verfOgen, ermoglicht, unteran- 
derem einem Teilnehmer, der ein herkommliches 
Telefon (74, 85), das mit dem PSTN (106, 110) ver- 
bunden ist, und einen Rechner (76, 88) hat, der mit 
dem externen Datennetzwerk (116) getrennt ver- 
bunden ist; einem Teilnehmer, der nur ein her- 
kommliches Telefon (72) hat, das mit dem PSTN 
(104) verbunden ist; einem Teilnehmer, der nur ei- 
nen Rechner hat, der mit dem externen Datennetz- 
werk verbunden ist, wobei der Rechner nicht zur 
Abwicklung von Sprache ausgestattet ist; einem 
Teilnehmer, der nur einen mit dem externen Daten- 
netzwerk (116) verbundenen Rechner (80) hat, wo- 
bei der Rechner zur Abwicklung sowohl von 
Sprach- als auch DatenObertragungen in Bezug auf 
den jeweiligen Teilnehmer ausgestattet ist; einem 
Teilnehmer, der nur eine mit dem externen Netz- 
werk verbundene Einheit hat, wobei die Einheit nur 
fOr die Abwicklung von SprachQbertragungen in Be- 
zug auf den jeweiligen Teilnehmer ausgestattet ist; 
Oder einem Teilnehmer, der nur einen mit dem 
PSTN verbundenen Rechner hat, wobei der Rech- 
ner zur Abwicklung sowohl von Sprachals auch Da- 
tenObertragungen in Bezug auf das PSTN ausge- 
stattet ist. 

10. Konferenzsystem nach einem vorhergehenden An- 
spruch, wobei der Konferenzserver (30) Folgendes 
enthalt: 



Mittel, die den Konferenzserver mit einer Mo- 
dembank (1 09) verbinden, urn Sprach- und Da- 
tensignale zwischen den internen Konferenz- 
schaltungen (20a) und der Modembank und 
5 zwischen dem externen Datennetzwerk (116) 

und der Modembank weiterzuleiten. 

11. Konferenzsystem nach Anspruch 10, wobei der 
Konferenzserver (30) dazu dient, 

10 

Sprach- und Datensignale, die in der Modem- 
bank (109) von einem einzigen der Teilnehmer 
empfangen werden, zu trennen; 

is die getrennten Sprachsignale an die Konfe- 

renzschaltungen (20a) weiterzuleiten, urn sie 
an die Telefone aller Konferenzteilnehmer, die 
uber Telefone verfOgen, anzuschlieften; und 

20 einige der getrennten Datensignale selektiv an 

die Konferenzschaltungen weiterzuleiten, urn 
sie Teilnehmern bereitzustellen, deren Rech- 
ner direkt an das PSTN angeschlossen sind, 
und urn die verbleibenden getrennten Datensi- 

25 gnale an das externe Datennetzwerk weiterzu- 

leiten, urn sie Teilnehmern bereitzustellen, de- 
ren Rechner direkt an das externe Datennetz- 
werk angeschlossen sind. 

30 

Revendications 

1. Systeme de conference vocal et de donnees com- 
prenant: 

35 

un serveur de conf6rence (30) communiquant 
avec un systeme de gestion de conference 
(20a), ledit serveur de conference possedant 
des connections de commande (31 , 32) a la fois 

40 vers des circuits de conference et un reseau de 

communications de donnees, lesdites con- 
nexions de commande (32) autorisant ledit ser- 
veur de conference a diriger les transferts de 
signaux de donnees au travers dudit reseau de 

45 communications de donnees, lesdites donnees 

transferees comprenant des donnees repre- 
sentant des donnees d'images affichables et 
des donnees representant des demandes et 
des commandes fonctionnelles emises par des 

so participants individuels a la conference et ne- 

cessitant Tattention dudit systeme de gestion 
de conference, 

caracterise en ce que : 

55 

le systeme de conference vocal et de donnees 
est destine au reseau telephonique public com- 
mute (RTCP), ledit reseau RTCP (1) compor- 
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tant lesdits circuits de conference (2a) de facon 
entierement interne a celui-ci afin d'acheminer 
des signaux vocaux analogiques entre des uti- 
lisateurs finaux (5, 6) dudit reseau RTCP, et 
comportant egalement ledit systeme de gestion 5 
de conference (20a) de facon entierement in- 
terne a celui-ci afin de gerer des operations 
dans lesdits circuits de conference pour fournir 
les connections de conference entre lesdits uti- 
lisateurs finaux, 10 3. 

le serveur de conference fonctionnant comme 
une partie interne dudit RTCP, 

le reseau de communication de donnees (16) *5 
est exterieur audit reseau RTCP, 

lesdites connexions de commande autorisent 
ledit serveur de conference a diriger les trans- 
ferts de signaux de donnees au travers dudit 20 
r6seau de communications de donnees exte- 4. 
rieur en coordination avec une conference vo- 
cale etablie dans lesdits circuits de conference 
internes, lesdites donnees etant transferees en 
parallele au travers dudit reseau de donnees 25 
exferieur, entre les participants de ladite confe- 
rence vocale etablie poss6dant des con- 
nexions vers ledit reseau exterieur, en coordi- 
nation avec les signaux vocaux transmis entre 
lesdits participants au travers desdits circuits 30 
de conference internes, lesdites demandes 
fonctionnelles comprenant des demandes de 
transfert de donnees, acheminees depuis un dit 5. 
participant vers ledit serveur de conference, 
pour que ledit serveur de conference gere la 35 
repartition des donnees representant lesdites 
donnees d'images affichables, lesdites don- 
nees d'images ayant ete precedemment four- 
nies audit serveur de conference par un parti- 
cipant emettant la demande, aux autres dits *o 
participants, 6. 

lesdites connexions de commande et deman- 
des etant utilisees pour que ledit systeme de 
gestion de conference r6partisse lesdites don- 45 
nees representant lesdites donnees d'images 
affichables entre lesdits participants a la confe- 
rence en coordination avec une presentation 
vocale realisee par un participant demandeur, 
lesdites connexions de commande, demandes, so 
et instructions etant en outre utilisees pour que 
ledit systeme de gestion de conference execu- 
te des fonctions de commande par rapport a la 
fois auxdits circuits de conference internes et 
audit reseau de donnees exterieur. 55 

Systeme de conference vocal et de donnees selon 
la revendication 1, dans lequel lesdits moyens de 



connexions de commande (31) entre ledit serveur 
de conference (30) et lesdits deux circuits de con- 
ference (21 ) et systeme de gestion de conference 
(20a) sont utilises en reponse a certaines deman- 
des fonctionnelles, pour signaler audit systeme de 
gestion de conference de modifier des conditions 
de fonctionnement dans lesdits circuits de confe- 
rence. 

Systeme de conference selon la revendication 1 ou 
la revendication 2, dans lequel ledit serveur de con- 
ference (30) comprend en outre : 

un moyen sensible aux signaux recus d'un 
moyen de reconnaissance vocal afin de repartir 
des pages individuelles desdites donnees 
d'images affichables auxdits participants et 
d'executer d'autres fonctions se rapportant a la- 
dite conference vocale desdits participants. 

Systeme de conference selon la revendication 2, 
dans lequel lesdits signaux vers ledit systeme de 
gestion de conference (20a) pour modifier des con- 
ditions dans lesdits circuits de conference (21 ) sont 
utilises pour amener la connexion tefephonique 
d'un dit participant a la conference (5, 6) a §tre 
changee de facon a autoriser le participant respectif 
a changer entre des canaux cables et sans fits de 
connexion vers ledit r6seau RTCP (1) sans inter- 
rompre le cours d'une conference actuellement ac- 
tive. 

Systeme de conference selon la revendication 2, 
dans lequel lesdits signaux vers ledit systeme de 
gestion de conference (20a) pour modifier des con- 
ditions dans lesdits circuits de conference (21 ) sont 
utilises pour modifier le volume sonore actuelle- 
ment delivre a un recepteur tefephonique utilise par 
I'un desdits participants. 

Systeme de conference selon Tune quelconque des 
revendications precedentes, dans lequel lesdites 
connexions de commande (31) entre ledit serveur 
de conference (30) et lesdits circuits de conference 
(20a) sont utiles pour autoriser ledit serveur de con- 
ference a transferer des signaux representant des 
informations de voix et d'images affichables entre 
lesdits circuits internes de conference etun equipe- 
ment prenant en charge de la voix et une image af- 
fichable relfe audit reseau exterieur de donn6es 
(16), et egalement pour autoriser ledit serveur de 
conference a transferer des signaux repr6sentant 
des donnees d'images affichables entre lesdits 
equipements de prise en charge d'images afficha- 
bles connectes audit reseau de donnees exferieur, 
en autorisant ainsi que des connexions deficitaires 
d'utilisateurs finaux du reseau RTCP (1) vers ledit 
reseau de donn6es envoient et recoivent des don- 
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n£es relatives au reseau de donnees par Pinterme- 
diaire du reseau RTCP. 

7. Systeme de conference seion Tune quelconque des 
revendications precedentes, dans lequel ledit re- 5 
seau de communications de donnees exterieur (16) 
qui ne fait pas partie du systeme de conference et 
vers lequel le serveur de conference (30) possede 
une connexion de commande, est un reseau public 

de donnees qui est accessible a tout membre du 10 
public se conformant aux exigences du reseau res- 
pectif. 

8. Systeme de conference selon Tune quelconque des 
revendications precedentes, dans lequel ledit ser- 15 
veur de conference (30) et efficace pour elaborer 

un trajet de circuit entre lesdits circuits internes de 
conference (20a) dans ledit reseau RTCP (1 ) et ledit 
reseau de donnees exterieur (16) pour acheminer 
selectivement des signaux representant a la fois de 20 
la voix et des donnees entre lesdits circuits de con- 
ference et ledit reseau de donn6es exterieur. 

9. Systeme de conference selon la revendication 8, 
dans lequel ledit trajet de circuit forme par ledit ser- 25 
veur de conference (30) autorise ledit reseau RTCP 

(1 ) et le reseau de donnees exterieur (1 6) a prendre 
en charge des conferences entre les participants a 
la conference possedant differents equipements et 
differentes connexions de communication y com- 30 
pris un participant possedant un telephone classi- 
que (74, 85) relie au reseau RTCP (106, 110) et un 
ordinateur (76, 88) relies separement audit reseau 
de donn6es exterieur (116), un participant n'ayant 
qu'un telephone classique (72) relie au reseau 35 
RTCP (104), un participant n'ayant qu'un ordinateur 
relie audit reseau de donnees exterieur, ledit ordi- 
nateur n'etant pas equipe pour prendre en charge 
la voix, un participant ne possedant qu'un ordina- 
teur (80) relie audit reseau de donnees exterieur 
(1 1 6), ledit ordinateur etant equipe pour prendre en 
charge a la fois la voix et les transmissions de don- 
nees se rapportant au participant respectif, un par- 
ticipant ne possedant qu'un dispositif relie audit re- 
seau exterieur, ledit dispositif n'etant equipe que 45 
pour prendre en charge les communications voca- 
les se rapportant au participant respectif, ou bien 
un participant n'ayant qu'un ordinateur relie audit 
reseau RTCP, ledit ordinateur etant 6quipe pour 
prendre en charge a la fois la voix et les transmis- so 
sions de donnees par rapport audit reseau RTCP. 



(109) destinee a acheminer la voix et des si- 
gnaux de donnees entre lesdits circuits inter- 
nes de conference (20a) et ladite batterie de 
modulateurs-demodulateurs et entre ledit re- 
seau de donnees exterieur (116) et ladite bat- 
terie de modulateurs-demodulateurs. 

11. Systeme de conference selon la revendication 10, 
dans lequel ledit serveur de conference (30) est ef- 
ficace pour : 

separer les signaux de voix et de donnees re- 
cus dans ladite batterie de modulateurs-demo- 
dulateurs (109) provenant d'un seul dit partici- 
pant, 

acheminer les signaux vocaux separes vers 
lesdits circuits de conference (20a) en vue 
d'une connexion aux telephones de tous les 
participants a la conference munis de telepho- 
nes, et 

acheminer selectivement certains des signaux 
de donnees separ6s vers lesdits circuits de 
conference, en vue d'une delivrance aux parti- 
cipants ayant des ordinateurs directement re- 
lies audit reseau RTCP, et acheminer les si- 
gnaux de donnees separes restants vers ledit 
reseau de donnees exterieur en vue d'une de- 
livrance aux participants ayant des ordinateurs 
directement connectes audit reseau de don- 
nees exterieur. 



10. Systeme de conference selon Tune quelconque des 
revendications precedentes, dans lequel ledit ser- 
veur de conference (30) comprend : 55 

un moyen reliant ledit serveur de conference a 
une batterie de modulateurs-demodulateurs 
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Participants establish separate conference 
connections to PSTN and a shared server 
having no fink to PSTN conference 
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Participants converse over PSTN connection 
and exchange data via server. Since server is 
unlinked to voice conference, data interchange 
can neither be coordinated with nor affect voice; 
and data security may be costly to implement 



Figure 6 
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Participants estabfish separate conference 
connections to PSTN and a conference server 
managed and administered by the PSTN 
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Participants converse over PSTN connection 
and exchange data via server. Since server is 
linked to and part of voice conference center, 
data interchange can be coordinated with voice 
presentations of individual speakers, and even 
used to affect conditions in the voice 
transmission path. Data security may be 
provided by PSTN at relatively little cost 



19 



EP 0 847 178 B1 



Figure 6a 



Voice signals flow between 
conferees only through 
PSTN lines and circuits, 

including conference bridge 
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tf conference center 
equipped for speech 
recognition, spoken 
commands are detected and 
sent to conference manager 
or conference server 
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Image data flows from originating conferee 
to conference server to other conferees; 
remaining separate from voice throughout 
their handling to and from the server 
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Data signals may represent displayable 
images, visual cues to be displayed (e.g. bid 
to interrupt speaker), other cues, or service 
request changes to be implemented by the 
conference server either directly or through 
the conference manager 



Figure 7 



Conference Master sends page data 
to PSTN conference server, before or 
during conference 
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During conference, conference server 
distributes pages one at a time in response 
to cues from conference master (data or 
vocal cues) 




f 


If conferees have a 
they may modify pag 
upload it to serve 
distribute it 


rttaboration software, 
|e currently displayed; 
rand cue server to 
to the others 
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Conferees send requests and/or other 
displayable data, server reacts to requests 
and distributes other data 
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Figure 8 



( 55 

Participant sends request to server pertaining to ) 
operation in PSTN voice signal path V 
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If operation is permitted, server executes it or 
signals conference manager to do it 



J 



Server signals either acknowledgement of execution or 
"operation not permitted" indication to requestor 



57 

V 



Figures 



Participant signals request to speak 
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Conference sewer Bluminates icon associated 
with' request In page currently being displayed 
to all conferees 
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Figure 10 
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Participant requests modification of voice 
signals sent to him/her 
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Conference server forwards corresponding 
signal to conference manager 



V 
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Action implemented in voice path 
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